\abstract


Network testbeds have become a popular mechanism 
 for evaluation of research technologies
in networking, security, peer-to-peer, 
distributed systems and many other computer science fields.
Many network testbeds exist today and
 more are being actively developed.
Yet there is little data about how testbeds are being used, by whom, and
for what purpose. Such data would help build better testbeds and improve
testbed management and usage policies. 

This paper analyzes the usage datasets of five public network testbeds, 
 primarily
DETER and Utah Emulab testbeds and additionally Schooner, Planetlab and
StarBED. From the testbed logs about users, experimentation activity, 
 and project descriptions, along with a traditional 
 user survey about interactions with testbeds, 
  we observe several interesting testbed usage characteristics. 
Since experimentation inherently challenging
most testbed experiments start with small-scale studies 
 and then evolve into large-scale experiments. 
We also find testbed usage has steadily increased 
 over time due to increase in membership 
 but experiment size in terms of nodes 
  has decreased due to contention of 
  testbed resources. 
Also, we discuss how the current lack of 
 experimentation tools and services  
 negatively impacts the use of network 
 testbeds for research. 
 



\iffalse
Network testbeds have become a popular tool for evaluation of 
research solutions in networking, security, peer-to-peer, 
distributed systems and many other computer science fields.
Many network testbeds exist today and
 more are being actively developed.
Yet there is little data about how testbeds are being used, by whom, and
for what purpose. Such data would help build better testbeds and improve
testbed management and use policies. Understanding points of
satisfaction and problems for testbed users would also help testbed
developers direct their efforts to better serve the user community.

This paper analyzes usage of several public network testbeds, primarily
DETER and Utah Emulab testbeds and additionally Schooner, Planetlab and
StarBED. From testbed log data about users and experiments, and from
a user survey about their satisfaction with testbeds we draw the following observations: (1) 
Most testbed experiments are small and short, and many have simple topologies,
yet they can yield sufficient data to result in a research publication, (2) Testbed use
increases over time due to increase in membership but experiment sizes and 
durations remain steady, (3) Features of 
user interaction with testbeds exhibit a wide range of values and follow a heavy-tailed 
distribution, (4) Testbeds need to improve their usability, fidelity and repeatability to 
stay competitive in experimentation market. 
\fi 

